"CharityEngine" (Batch Computation Provider)

Details

Charity Engine is a container-based grid computing service that runs jobs using idle processor time on a worldwide network of desktop computers.
To configure the "CharityEngine" batch computation provider for use in the Wolfram Language, follow the instructions in the Set Up the Charity Engine Batch Computation Provider workflow.
The "CharityEngine" batch computation provider stores input and output data for jobs using the file hosting system provided by Charity Engine.
The "CharityEngine" batch computation provider supports running jobs in Linux containers only.
The "CharityEngine" batch computation provider does not currently support Wolfram Language GPU computation.

Environment Properties

The following properties are supported in a RemoteBatchSubmissionEnvironment object for the "CharityEngine" provider:
  • "ServiceObject"AutomaticServiceObject for "CharityEngine" service
    "UseOwnDevices"Falsewhether to restrict job scheduling to hosts owned by the current user
    • If the "ServiceObject" property is omitted (or is set to Automatic), a service object will be automatically constructed and connected with ServiceConnect["CharityEngine"].
    If the "ServiceObject" property is set to "New", a new service connection will be created with ServiceConnect["CharityEngine","New"], disregarding any saved connections or credentials.
    A Charity Engine "authenticator" string is required to connect a "CharityEngine" service object. The Set Up the Charity Engine Batch Computation Provider workflow provides instructions for creating a Charity Engine account and obtaining an authenticator.

    Job Settings

    The following settings are supported by the RemoteProviderSettings option when using the "CharityEngine" provider:
  • "InstanceType""C.2x2"Charity Engine instance-type string
    "UseOwnDevices"Inheritedwhether to restrict job scheduling to hosts owned by the current user
    • The following basic instance types are supported by Charity Engine:
  • "C.2x2"2 vCPUs; 2GiB RAM
    "C.2x4"2 vCPUs; 4GiB RAM
    "C.4x4"4 vCPUs; 4GiB RAM
    • Additional instance types beyond those listed may be supported; check the Charity Engine website for a current list. Alternatively, a current list may be obtained using the "InstanceTypes" request of the "CharityEngine" service connection. »
    The value Inherited for the "UseOwnDevices" setting corresponds to the value specified in the RemoteBatchSubmissionEnvironment used to submit the job.
    When using the "CharityEngine" provider, the value of the TimeConstraint option of RemoteBatchSubmit and RemoteBatchMapSubmit must be between 1 and 8 hours, with a default of 8 hours.
    The "CharityEngine" provider does not support automatic forwarding of cloud credentials with the ForwardCloudCredentials option. »

    Job Statuses

    The following are possible values of the "JobStatus" job property when using the "CharityEngine" provider, listed in the order through which a typical job will pass:
  • "Provisioning"the job is queued and waiting to be scheduled to a host
    "Downloading"the job has been scheduled to a host and its container image and input files are being downloaded
    "Running"the job container has started and is evaluating the job code
    "Completed"the job's execution has succeeded and its output has been uploaded
    "Failed"the job's execution has failed
    "Aborted"the job was aborted or timed out before it could complete

    • Job Properties

    When using the "CharityEngine" provider, the following properties are available from "Single"-type job objects, in addition to the standard properties supported by RemoteBatchJobObject:
  • "JobLog"console logs from the job container
    "ProviderJobID"Charity Engine-provided unique identifier for the job
    • When using the "CharityEngine" provider, the following properties are available from "Array"-type job objects, in addition to the standard properties supported by RemoteBatchJobObject:
  • "ProviderChildJobIDs"Charity Engine-provided unique identifiers for each array child job
    • When using the "CharityEngine" provider, the following properties are available from "ArrayChild"-type job objects in addition to the standard properties supported by RemoteBatchJobObject:
  • "JobLog"console logs from the child job container
    "ProviderJobID"Charity Engine-provided unique identifier for the child job
    • The values of the "ProviderJobID" and "ProviderJobIDs" properties identify jobs to Charity Engine, in contrast to the "JobUUID" property, which identifies a job within the Wolfram System.
    Charity Engine expires and deletes the metadata of a completed ("Completed", "Failed" or "Aborted") job 14 days after submission, and that of an incomplete ("Provisioning", "Download" or "Running") job 28 days after submission.
    RemoteBatchJobObject expressions representing jobs that have expired cannot be queried for status or output and will not be listed by RemoteBatchJobs.

    Examples

    open allclose all

    Basic Examples  (2)

    Create an "CharityEngine" RemoteBatchSubmissionEnvironment object:

    Submit a job using the created environment:

    Query the job’s status:

    Query the job’s status again, after it has completed:

    Download the job’s output:

    Submit a job using an implicitly created submission environment:

    Job Settings  (1)

    "InstanceType"  (1)

    Instruct Charity Engine to run a submitted job using a "C.4x4" instance with 4 vCPUs available:

    Job Properties  (2)

    "Single" Jobs  (1)

    Submit a batch job to Charity Engine using RemoteBatchSubmit:

    Obtain the console output from the job container:

    "Array" and "ArrayChild" Jobs  (1)

    Submit a batch array job to Charity Engine using RemoteBatchMapSubmit:

    Obtain the Charity Engine job identifiers for each array child job:

    Obtain a RemoteBatchJobObject expression representing the first child job:

    Obtain the console output from the child job container:

    Properties & Relations  (1)

    A list of currently available Charity Engine instance types can be obtained via the "CharityEngine" service connection:

    Possible Issues  (1)

    The "CharityEngine" provider does not support automatic forwarding of cloud credentials: